1. Field of the Invention
The present invention relates to an imaging device, an image processing device, and an imaging method for eliminating the effects of a fixed pattern noise of an image sensor when carrying out shooting a plurality of times from commencement of shooting until completion, and combining a plurality of acquired image data.
2. Description of the Related Art
Conventionally, with a single lens reflex type imaging device, observing a subject image has been performed using an optical viewfinder. However, imaging devices for observing a subject image not through an optical viewfinder but by live view display for displaying an image that has been read from an image sensor using a liquid crystal monitor or the like are known. Also, instead of an optical viewfinder, imaging devices that display an image, using live view display, using an electronic viewfinder that is separate from a liquid crystal monitor, and are capable of switching between the liquid crystal monitor and the electronic viewfinder, are available on the market.
It was not conventionally possible to perform readout of image signals from an image sensor during exposure, at the time of a prolonged exposure such as bulb shooting, with either an optical viewfinder or an electronic viewfinder. This meant that the photographer was not able to confirm subject state or exposure state, and confirmation of an image was carried out after completion of shooting. As a result, exposure setting and exposure time were estimated by the photographer taking the photograph from brightness of the subject etc. to determine starts and completion of exposure, and it was not a simple matter for the photographer to acquire a desired taken image without shooting failures due to under exposure or overexposure.
Imaging devices have therefore been proposed for reading out an image signal from an image sensor at specified time intervals, and displaying images obtained by simple accumulative addition every time this image signal is read out from the image sensor on a liquid crystal monitor. For example, according to the imaging device disclosed in Japanese patent laid open number 2005-117395, it is possible to reduce failures of shooting by displaying current progress of an exposure at the time of long exposure shooting such as bulb shooting. Also, an imaging device for generating a bulb shooting image by reading out an image signal continuously from an image sensor and carrying out comparatively bright combination (combining means for comparing brightness levels for every pixel of image data, selecting brighter pixels to be reflected in a resulting combination) is proposed in Japanese patent number 4148586.
With bulb shooting, shooting is carried out using exposure for a long time spanning from a few seconds to a few minutes. As a characteristic of the image sensor, dark current components occur in photodiodes constituting pixels of the image sensor at the time of long time exposure, giving rise to fixed pattern noise. The extent to which dark current arises varies for every pixel, and increases in proportion to exposure time. There is also a characteristic that current increases as image sensor temperature rises, and appears in an image as defect noise or image density irregularities.
This fixed pattern noise due to dark current is only dependent on image sensor temperature at the time of exposure, and exposure time, irrespective of whether shooting in an exposed state or shooting in a light shielded state. With digital cameras that are currently on the market, therefore, at the time of bulb shooting after the user has taken a picture (taken image is made a bright image) a light shielding image is automatically taken at the same shutter speed, and FPN (Fixed Pattern Noise) cancellation processing to correct the fixed pattern noise is carried out by carrying out subtraction processing for the bright image data and the light shielded image data, in an image processing circuit after the image sensor.
With an imaging device described in the above-described related literature, a dark image is taken after the bright image has been taken. Dark current, which is a cause of fixed pattern noise, increases as temperature of the image sensor increases. This means that if image sensor temperature is reduced during shooting, it will not be possible to sufficiently correct fixed pattern noise with a dark image that has been acquired after bright image shooting. It is therefore being considered to shoot dark images before and after bright image shooting, and monitor the temperature of the image sensor at the time of shooting the respective dark images, and to perform correction by selecting a dark image that was acquired when temperature was high to select an image that is suitable for FPN cancellation.
However, depending on the image sensor, variations within the imaging area, namely, FPN (fixed pattern noise) caused by dark current (dark current shading), become smaller as temperature increases. Dark current shading arises because in a process of creating photodiodes (PD) that constitute pixels of the image sensor, it is not possible to create PD having uniform characteristics at the center and the periphery of the sensor, and because there is inconsistency in the extent to which dark current arises at the center portions and at the periphery.
Dark current itself, being the FPN, becomes larger as temperature of the center and the periphery of the image sensor increases. Generally, with the image sensor 50, as shown in FIG. 8C, in addition to an effective pixel region 51 for acquiring image output as a result of receiving incident light of an optical image on the image sensor, OB (Optical Black) pixel regions (horizontal OB pixels 53 and vertical OB pixels 54) that are in a physically light shielded state, are arranged at the periphery of the effective pixel region. Output signals of the OB pixel regions are detected as a representative value of dark current output of the image sensor effective pixels (reference OB output).
However, it is common practice for read out signals to be subjected to OB black processing with average output of OB pixels that are arranged at the periphery of the sensor as a reference OB output for dark output, in a sensor internal circuit or a DSP (digital signal processor) disposed after the sensor. For this reason, with a general image sensor 50, namely an image sensor 50 having dark current shading where dark current at the periphery becomes larger than at the center, as shown in FIG. 8A, in-plane variations arise such that output of a dark image at the center of the sensor becomes smaller as temperature increases.
On the other hand, for pixel defects, since there is a tendency for pixel defects to increase as temperature increases, as shown in FIG. 8B, defective pixels can be corrected to a certain extent of brightness even if a dark image for a higher temperature is used, in accordance with results of temperature detection before commencement and upon completion of shooting. However, for correction of in-plane variations using this method (dark color and shading), it is not possible to optimally correct the in-plane variations (dark current shading).